1. Field
One or more embodiments relate to a motion capture apparatus and method, and more particularly, to an apparatus and method for generating real-time motion for a three dimensional (3D) virtual character with a minimized number of sensors.
2. Description of the Related Art
As various types of game consoles have been widely distributed, interest has been aroused in game console interfaces that can be used to control movements of characters in games. For example, devices such as joysticks or mice are generally used to control the movements of displayed game characters, but more recently there has been an introduction of interfaces which control the movements of a displayed virtual character corresponding to the physical movements of a user.
To generate such movements of a three-dimensional (3D) virtual character corresponding to motions of a user, a skeleton model with given lengths and virtual joints is defined, with the virtual joints having the same degree of freedom as human joints. Here, infrared markers or inertial sensors are attached to each joint of a human body to measure the movement angle of the respective joint. Subsequently the location (x, y, z) and angles (φ, θ, ψ) of each joint are calculated by way of forward kinematics. These calculated locations and angles can then be used to define movement of the 3D virtual character, which may be referred to as avatars, for example.
FIG. 20 illustrates such a conventional method of capturing and demonstrating motion of an individual. In operation S2000, sensor values are captured through N multiple sensors positioned at N body portions of the individual, as noted above the sensors could be inertial or infrared markers. In operation 2010, locations and rotational angles are then extracted from the sensed movement or position for each of the N body portions. Here, primarily only the positions and rotational angles of each body portion having a sensor may be determined, without estimation of positions or rotation angles, resulting in the number of required sensors being equal to the number of joints of the body being modeled. Lastly, in operation S2020, the extracted locations and rotation angles are used to generate a virtual character.
International Patent Application WO 2007/058526, the disclosure of which is incorporated herein, demonstrates an example inertial sensor that can detect position and orientation/rotational changes, e.g., by twice integrating the detected free acceleration of the inertial sensor to yield a position value. The inertial sensor can be recalibrated by adding a magnetic field sensor and using detected magnetic field information for calibrating the inertial sensor. Here, a transmitter broadcasts one or more magnetic pulses in different spatial directions. For example, the different magnetic pulses can be generated by three differently oriented coil systems, each generating a magnetic field along respective different spatial directions. The magnetic field detector is used for periodically providing different position/orientation information relative to a positioning of the transmitter that can be used to recalibrate the inertial measurement unit. Similar to the method of FIG. 20, the position and orientation changes of each sensor would be matched to each body part being modeled. Also see Inertial Sensing of Human Movement, H. J. Luinge, PhD thesis, Dec. 2002, which discusses such inertial sensors.
Thus, with numerous sensors, a sufficient amount of information can be obtained with regard to a given human skeleton model, and the human motion can thus be represented with comparatively high accuracy on a 3D virtual character through defined equations. However, to this end, a great number of sensors need to be attached to the body. In addition, with so many sensors it becomes difficult to institute corrections after an initial measurement is made. Moreover in terms of system costs, such an arrangement is typically not appropriate as an interface for home game consoles, such as those providing 3D games, since expensive equipment is required to implement the information collection and analysis. For example, in general, such systems require optical motion capture equipment such as OptoTrak (NDI) for high-accuracy motion capture. In using such optical motion capture equipment, an externally installed camera system needs to be large enough to capture all markers attached to the body, or a numerous number of cameras are required. Hence, for wider applications, it is more desirable to effectively create motion of a 3D virtual character from the measured body movement using less sensor information, if possible.
There have been studies on the representation of human movement through the use of fewer sensors. In one configuration of Hiroyuki Ukida et. al, Human Motion Capture System Using Color Markers and Silhouette, IMTC 2006, Italy, Apr. 2006, such representation requires locations and angles of movement of a body be extracted from external coordinates or markers. Consequently, there is a substantial amount of external equipment that is needed to ensure the view of the camera and to accommodate the required parallel processing of data and image information, as a similar substantial amount of information is required for generating actual motions. In another configuration of Ludovic Saint-Bauzel et. al, Real-time human posture observation from a small number of joint, IROS 2007, San Diego, Oct. 2007, a machine learning technique is implemented which only enables the distinguishing of given gestures such as a user sitting or standing. However, these example techniques are not efficient in representing general human movements. Further, such techniques are not appropriate for more user friendly application platforms such as home computers or game consoles.
Accordingly, as conventional systems require as much information as possible to represent more natural movements of a virtual character, a significant number of sensors need to be attached on a body for sampling, which results in increased costs, and requires additional computational power, while limiting correction capabilities after the initial measurements are made. Such an arrangement is accordingly not appropriate for most applications, including for generating a virtual character to be used as an interface of a home computer or game console. Conversely, a system that requires fewer components and sensors can only provide a limited amount of information, which again makes the application less desirable as only a limited number of movements can be detected, and thus only a limited number of gestures could be reflected in a displayed avatar.